{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VPau94vm3IpB"
   },
   "source": [
    "# Lesson 3: Loading ML Models with Different Data Types\n",
    "\n",
    "In this lab, you will load ML models in different datatypes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XwU9odrv4uph"
   },
   "source": [
    "- Load the Dummy Model from the helper file.\n",
    "- To access the `helper.py` file, you can click `File --> Open...`, on the top left."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 46
   },
   "outputs": [],
   "source": [
    "#123456789#123456789#123456789#123456789#123456789#123456789#123456789"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29,
    "id": "sNkJ_nJd4F7F"
   },
   "outputs": [],
   "source": [
    "from helper import DummyModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 433,
     "status": "ok",
     "timestamp": 1699380416965,
     "user": {
      "displayName": "Younes Belkada",
      "userId": "15414910276690549281"
     },
     "user_tz": -60
    },
    "height": 29,
    "id": "xhMZuiiE4tR5",
    "outputId": "2f36a01c-fb85-49f6-ee0d-36b5777167b4"
   },
   "outputs": [],
   "source": [
    "model = DummyModel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dJoz_iQM5bgV"
   },
   "source": [
    "- Create a function to inspect the data types of the parameters in a model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 293,
     "status": "ok",
     "timestamp": 1699380006138,
     "user": {
      "displayName": "Younes Belkada",
      "userId": "15414910276690549281"
     },
     "user_tz": -60
    },
    "height": 63,
    "id": "KVci0pER5ccu",
    "outputId": "7f4a6fc1-b168-42a2-baf4-ff5f00fba731"
   },
   "outputs": [],
   "source": [
    "def print_param_dtype(model):\n",
    "    for name, param in model.named_parameters():\n",
    "        print(f\"{name} is loaded in {param.dtype}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print_param_dtype(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nN792T445lG2"
   },
   "source": [
    "## Model Casting: `float16`\n",
    "\n",
    "- Cast the model into a different precision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 46,
    "id": "-_qUBPhy53L5"
   },
   "outputs": [],
   "source": [
    "# float 16\n",
    "model_fp16 = DummyModel().half()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Inspect the data types of the parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "print_param_dtype(model_fp16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "model_fp16"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zSlCqQL274P_"
   },
   "source": [
    "- Run simple inference using model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "dummy_input = torch.LongTensor([[1, 0], [0, 1]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 356
    },
    "executionInfo": {
     "elapsed": 233,
     "status": "error",
     "timestamp": 1699380421192,
     "user": {
      "displayName": "Younes Belkada",
      "userId": "15414910276690549281"
     },
     "user_tz": -60
    },
    "height": 46,
    "id": "5wcBVd_N70oH",
    "outputId": "e5f36295-2086-4a67-94d1-49e438d1c17d"
   },
   "outputs": [],
   "source": [
    "# inference using float32 model\n",
    "logits_fp32 = model(dummy_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29,
    "id": "B3I1G2zy8d1x"
   },
   "outputs": [],
   "source": [
    "logits_fp32"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 114,
    "id": "siPMcJRX84WZ"
   },
   "outputs": [],
   "source": [
    "# inference using float16 model\n",
    "try:\n",
    "    logits_fp16 = model_fp16(dummy_input)\n",
    "except Exception as error:\n",
    "    print(\"\\033[91m\", type(error).__name__, \": \", error, \"\\033[0m\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "D8U4TCxb9NR0"
   },
   "source": [
    "## Model Casting: `bfloat16`\n",
    "\n",
    "#### Note about deepcopy\n",
    "- `copy.deepcopy` makes a copy of the model that is independent of the original.  Modifications you make to the copy will not affect the original, because you're making a \"deep copy\".  For more details, see the Python docs on the [copy][https://docs.python.org/3/library/copy.html] library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "from copy import deepcopy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "model_bf16 = deepcopy(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "model_bf16 = model_bf16.to(torch.bfloat16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "print_param_dtype(model_bf16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "logits_bf16 = model_bf16(dummy_input)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "krxuhBKx9tcl"
   },
   "source": [
    "- Now, compare the difference between `logits_fp32` and `logits_bf16`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 2,
     "status": "ok",
     "timestamp": 1699380547898,
     "user": {
      "displayName": "Younes Belkada",
      "userId": "15414910276690549281"
     },
     "user_tz": -60
    },
    "height": 80,
    "id": "SkasCWBH-Dm5",
    "outputId": "193b71af-d66a-4030-ff62-e86ebd062ead"
   },
   "outputs": [],
   "source": [
    "mean_diff = torch.abs(logits_bf16 - logits_fp32).mean().item()\n",
    "max_diff = torch.abs(logits_bf16 - logits_fp32).max().item()\n",
    "\n",
    "print(f\"Mean diff: {mean_diff} | Max diff: {max_diff}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sdN9QpMN-mUw"
   },
   "source": [
    "## Using Popular Generative Models in Different Data Types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GmFTCvcNANQX"
   },
   "source": [
    "- Load [Salesforce/blip-image-captioning-base](https://huggingface.co/Salesforce/blip-image-captioning-base) to perform image captioning.\n",
    "\n",
    "#### To get the sample code that Younes showed:\n",
    "- Click on the \"Model Card\" tab.\n",
    "- On the right, click on the button \"<> Use in Transformers\", you'll see a popup with sample code for loading this model.\n",
    "\n",
    "```Python\n",
    "# Load model directly\n",
    "from transformers import AutoProcessor, AutoModelForSeq2SeqLM\n",
    "\n",
    "processor = AutoProcessor.from_pretrained(\"Salesforce/blip-image-captioning-base\")\n",
    "model = AutoModelForSeq2SeqLM.from_pretrained(\"Salesforce/blip-image-captioning-base\")\n",
    "```\n",
    "\n",
    "- To see the sample code with an example, click on \"Read model documentation\" at the bottom of the popup.  It opens a new tab.\n",
    "  https://huggingface.co/docs/transformers/main/en/model_doc/blip#transformers.BlipForConditionalGeneration\n",
    "- On this page, scroll down a bit, past the \"parameters\", section, and you'll see \"Examples:\"\n",
    "\n",
    "```Python\n",
    "from PIL import Image\n",
    "import requests\n",
    "from transformers import AutoProcessor, BlipForConditionalGeneration\n",
    "\n",
    "processor = AutoProcessor.from_pretrained(\"Salesforce/blip-image-captioning-base\")\n",
    "model = BlipForConditionalGeneration.from_pretrained(\"Salesforce/blip-image-captioning-base\")\n",
    "\n",
    "url = \"http://images.cocodataset.org/val2017/000000039769.jpg\"\n",
    "image = Image.open(requests.get(url, stream=True).raw)\n",
    "text = \"A picture of\"\n",
    "\n",
    "inputs = processor(images=image, text=text, return_tensors=\"pt\")\n",
    "\n",
    "outputs = model(**inputs)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81,
     "referenced_widgets": [
      "426d8aec5ed44782963707817195db5d",
      "51a84bbc69874d5bb065701aba6c5a22",
      "738c5fefbd4842ebacf8d0248a4e5bdd",
      "65b5176702244e338c1120e702f3ca96",
      "cebfa21c01524fa89ea124485867d8cc",
      "bc254bc2525f43f7bea70b2f6f515288",
      "f3b1dc11f2a44da3a5db8499c1401288",
      "b3140b792b3c47c68907d7a86259e389",
      "5070aad508574d0da4f667885e9f8e6d",
      "ae278dae0c484e85831aafd7a4ad6484",
      "1b3d91e17c0a4435860477f0593fc014",
      "b41130569642470ca5efb531e7652e17",
      "afac2f5a68c946b387f30e914a62729d",
      "4fecdb69b5f942449767d63c2bbb2dc9",
      "a3333106642d40d9a3066e9e2d79a650",
      "0a1f3685b7264cc595355cd906cb0e4d",
      "9dda2b50e90446438835f902b2b9d4b1",
      "21c6e937405d42048aa5607dbe779465",
      "47d756b3efba43a8a45b48cac04cf929",
      "d0d8b31e3c6b42318d804cc67a5aacea",
      "f3ac895d29454e58bffbde53fc00a05e",
      "b076ffc6b36a46c684557086a8c16582"
     ]
    },
    "executionInfo": {
     "elapsed": 14495,
     "status": "ok",
     "timestamp": 1699381117014,
     "user": {
      "displayName": "Younes Belkada",
      "userId": "15414910276690549281"
     },
     "user_tz": -60
    },
    "height": 29,
    "id": "sJ8__4_sAMKT",
    "outputId": "e18cad23-6bd5-4bf8-d059-586f8b772c9f"
   },
   "outputs": [],
   "source": [
    "from transformers import BlipForConditionalGeneration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "model_name = \"Salesforce/blip-image-captioning-base\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "model = BlipForConditionalGeneration.from_pretrained(model_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 63,
    "id": "Ptm1kGMTA8dB"
   },
   "outputs": [],
   "source": [
    "# inspect the default data types of the model\n",
    "\n",
    "# print_param_dtype(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_YYQNKfnAgsb"
   },
   "source": [
    "- Check the memory footprint of the model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "fp32_mem_footprint = model.get_memory_footprint()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 80
   },
   "outputs": [],
   "source": [
    "print(\"Footprint of the fp32 model in bytes: \",\n",
    "      fp32_mem_footprint)\n",
    "print(\"Footprint of the fp32 model in MBs: \", \n",
    "      fp32_mem_footprint/1e+6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N7GgA7mnAuXV"
   },
   "source": [
    "- Load the same model in `bfloat16`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 9,
     "status": "ok",
     "timestamp": 1699381195895,
     "user": {
      "displayName": "Younes Belkada",
      "userId": "15414910276690549281"
     },
     "user_tz": -60
    },
    "height": 80,
    "id": "9v4XJcvLAt5o",
    "outputId": "abbc3fd9-f8a9-4c5e-dd3d-61ddcf316d27"
   },
   "outputs": [],
   "source": [
    "model_bf16 = BlipForConditionalGeneration.from_pretrained(\n",
    "                                               model_name,\n",
    "                               torch_dtype=torch.bfloat16\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29,
    "id": "pVnUmBBABJxB"
   },
   "outputs": [],
   "source": [
    "bf16_mem_footprint = model_bf16.get_memory_footprint()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 6,
     "status": "ok",
     "timestamp": 1699381416456,
     "user": {
      "displayName": "Younes Belkada",
      "userId": "15414910276690549281"
     },
     "user_tz": -60
    },
    "height": 114,
    "id": "6Co7vpcYBl60",
    "outputId": "c282be1e-95b9-4591-f56f-945a2587dd1e"
   },
   "outputs": [],
   "source": [
    "# Get the relative difference\n",
    "relative_diff = bf16_mem_footprint / fp32_mem_footprint\n",
    "\n",
    "print(\"Footprint of the bf16 model in MBs: \", \n",
    "      bf16_mem_footprint/1e+6)\n",
    "print(f\"Relative diff: {relative_diff}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GeGHdNROBqPa"
   },
   "source": [
    "### Model Performance: `float32` vs `bfloat16`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eT7n0ybhCNOk"
   },
   "source": [
    "- Now, compare the generation results of the two model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 8317,
     "status": "ok",
     "timestamp": 1699381678824,
     "user": {
      "displayName": "Younes Belkada",
      "userId": "15414910276690549281"
     },
     "user_tz": -60
    },
    "height": 29,
    "id": "m3z3N6k3CN_B",
    "outputId": "b255e213-ccb8-43f9-9fb9-d31558e885b2"
   },
   "outputs": [],
   "source": [
    "from transformers import BlipProcessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "processor = BlipProcessor.from_pretrained(model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Load the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 148
   },
   "outputs": [],
   "source": [
    "from helper import load_image, get_generation\n",
    "from IPython.display import display\n",
    "\n",
    "img_url = 'https://storage.googleapis.com/\\\n",
    "sfr-vision-language-research/BLIP/demo.jpg'\n",
    "\n",
    "image = load_image(img_url)\n",
    "display(image.resize((500, 350)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 80
   },
   "outputs": [],
   "source": [
    "results_fp32 = get_generation(model, \n",
    "                              processor, \n",
    "                              image, \n",
    "                              torch.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "print(\"fp32 Model Results:\\n\", results_fp32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 80
   },
   "outputs": [],
   "source": [
    "results_bf16 = get_generation(model_bf16, \n",
    "                              processor, \n",
    "                              image, \n",
    "                              torch.bfloat16)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "print(\"bf16 Model Results:\\n\", results_bf16)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Default Data Type\n",
    "\n",
    "- For Hugging Face Transformers library, the deafult data type to load the models in is `float32`\n",
    "- You can set the \"default data type\" as what you want."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 212,
     "status": "ok",
     "timestamp": 1699382665501,
     "user": {
      "displayName": "Younes Belkada",
      "userId": "15414910276690549281"
     },
     "user_tz": -60
    },
    "height": 46,
    "id": "7FlJlxH2GJzQ",
    "outputId": "95cf9a07-c366-42e4-8e62-1858a0dd3703"
   },
   "outputs": [],
   "source": [
    "desired_dtype = torch.bfloat16\n",
    "torch.set_default_dtype(desired_dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "dummy_model_bf16 = DummyModel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "print_param_dtype(dummy_model_bf16)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Similarly, you can reset the default data type to float32."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "torch.set_default_dtype(torch.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "print_param_dtype(dummy_model_bf16)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Note\n",
    "- You just used a simple form of quantization, in which the model's parameters are saved in a more compact data type (bfloat16).  During inference, the model performs its calculations in this data type, and its activations are in this data type.\n",
    "- In the next lesson, you will use another quantization method, \"linear quantization\", which enables the quantized model to maintain performance much closer to the original model by converting from the compressed data type back to the original FP32 data type during inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "-_qUBPhy53L5",
    "N7GgA7mnAuXV"
   ],
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "0a1f3685b7264cc595355cd906cb0e4d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1b3d91e17c0a4435860477f0593fc014": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "21c6e937405d42048aa5607dbe779465": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "426d8aec5ed44782963707817195db5d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_51a84bbc69874d5bb065701aba6c5a22",
       "IPY_MODEL_738c5fefbd4842ebacf8d0248a4e5bdd",
       "IPY_MODEL_65b5176702244e338c1120e702f3ca96"
      ],
      "layout": "IPY_MODEL_cebfa21c01524fa89ea124485867d8cc"
     }
    },
    "47d756b3efba43a8a45b48cac04cf929": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4fecdb69b5f942449767d63c2bbb2dc9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_47d756b3efba43a8a45b48cac04cf929",
      "max": 989820849,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d0d8b31e3c6b42318d804cc67a5aacea",
      "value": 989820849
     }
    },
    "5070aad508574d0da4f667885e9f8e6d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "51a84bbc69874d5bb065701aba6c5a22": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bc254bc2525f43f7bea70b2f6f515288",
      "placeholder": "​",
      "style": "IPY_MODEL_f3b1dc11f2a44da3a5db8499c1401288",
      "value": "Downloading (…)lve/main/config.json: 100%"
     }
    },
    "65b5176702244e338c1120e702f3ca96": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ae278dae0c484e85831aafd7a4ad6484",
      "placeholder": "​",
      "style": "IPY_MODEL_1b3d91e17c0a4435860477f0593fc014",
      "value": " 4.56k/4.56k [00:00&lt;00:00, 59.3kB/s]"
     }
    },
    "738c5fefbd4842ebacf8d0248a4e5bdd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b3140b792b3c47c68907d7a86259e389",
      "max": 4563,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5070aad508574d0da4f667885e9f8e6d",
      "value": 4563
     }
    },
    "9dda2b50e90446438835f902b2b9d4b1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a3333106642d40d9a3066e9e2d79a650": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f3ac895d29454e58bffbde53fc00a05e",
      "placeholder": "​",
      "style": "IPY_MODEL_b076ffc6b36a46c684557086a8c16582",
      "value": " 990M/990M [00:04&lt;00:00, 219MB/s]"
     }
    },
    "ae278dae0c484e85831aafd7a4ad6484": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "afac2f5a68c946b387f30e914a62729d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9dda2b50e90446438835f902b2b9d4b1",
      "placeholder": "​",
      "style": "IPY_MODEL_21c6e937405d42048aa5607dbe779465",
      "value": "Downloading pytorch_model.bin: 100%"
     }
    },
    "b076ffc6b36a46c684557086a8c16582": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b3140b792b3c47c68907d7a86259e389": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b41130569642470ca5efb531e7652e17": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_afac2f5a68c946b387f30e914a62729d",
       "IPY_MODEL_4fecdb69b5f942449767d63c2bbb2dc9",
       "IPY_MODEL_a3333106642d40d9a3066e9e2d79a650"
      ],
      "layout": "IPY_MODEL_0a1f3685b7264cc595355cd906cb0e4d"
     }
    },
    "bc254bc2525f43f7bea70b2f6f515288": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cebfa21c01524fa89ea124485867d8cc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d0d8b31e3c6b42318d804cc67a5aacea": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f3ac895d29454e58bffbde53fc00a05e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f3b1dc11f2a44da3a5db8499c1401288": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
